Fix LCAO Hermitian matrix handling and EXX little-group density restoration - #7923

Reminder

• I have read AGENTS.md and docs/developers_guide/agent_governance.md.
• I have linked an issue or explained why this PR does not need one.
• I have added adequate unit tests and/or case tests, or explained why not.
• I have listed the exact verification commands run and their results.
• I have described user-visible behavior changes, including INPUT parameter changes.
• I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
• I have requested any needed governance exception below.

Linked Issue

Related to #7914. This PR corrects authoritative-upper-triangle handling in the separate native ELPA and cuSolver implementations, and fixes an EXX little-group density-restoration issue exposed by enforcing that matrix contract. No separate issue is required.

The genelpa transform and fallback corrections from #7914 are now included through upstream develop, with TaoXia's original attribution preserved. They are no longer a pending dependency.

Unit Tests and/or Case Tests for my changes

Commands run

For the fresh six-way experiment, each diagnostic variant was rebuilt from commit 75ee7493a, then run in a separate case directory:

export OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1
export UCX_TLS=^xpmem

cmake --build "$BUILD_DIR" -j8 --target abacus_std_para
"$BUILD_DIR/abacus_std_para" --version

# VARIANT_BINARY identifies the corresponding rebuilt diagnostic executable.
timeout 180 mpirun -np 1 "$VARIANT_BINARY"

The experiment crosses two controlled factors:

• little-group averaging: off / on;
• solver path: old PZGEMM transform control / corrected PZHEMM / ScaLAPACK.

The old-transform control changes only the Cholesky-branch hemm('L','U',A,B) operation back to gemm('C','N',A,B) on the same current baseline. It does not revert the whole repository. All recorded ELPA decomposition states were 1; the fallback branch was not exercised in this experiment.

Both ELPA controls use the same diagnostic one-stage setting to avoid the previously observed installed two-stage library failure. This setting is not part of the production changes in the PR.

Result summary

All six calculations completed successfully on 2026-09-14.

Without averaging, the maximum solver-dependent energy difference is 2.678 meV, while corrected PZHEMM and ScaLAPACK agree within 1.073e-9 eV.

After little-group averaging, all three paths agree within 5.33305e-10 eV, including the old PZGEMM control. They also give the same force absolute sum (0.000000 eV/Å) and stress absolute sum (2143.792554 kbar) at the output precision.

Matrix diagnostics were collected at the common solver entry point: 57 solves per case, 342 complete H/S and eigenpair records in total.

The raw matrix norms use ABACUS's internal matrix units. The normalized residual is

Here, $E$ is the diagonal matrix of computed eigenvalues, $H$ is the original full Hamiltonian, and $S_{U}$ is reconstructed from the authoritative upper triangle of $S$.

Independent NumPy Cholesky/eigvalsh calculations give eigenvalue disagreements no larger than 4.885e-15 in internal energy units after averaging.

Without averaging, corrected PZHEMM and ScaLAPACK solve the upper-triangle-completed Hermitian problem with residuals below 3.020e-16, but their residuals against the original full H are approximately 3.511e-4. After averaging, both the upper-triangle contract and the full-matrix eigenproblem are satisfied at roundoff.

Matched conditions and provenance:

• Source baseline: commit 75ee7493a.
• Hardware: Intel Xeon CPU Max 9470C CPU.
• All runs: one MPI rank, one OpenMP thread, one OpenBLAS thread.
• GNU toolchain, OpenMPI 5.0.10, ELPA 2026.02.001-2603, LibXC 7.0.0, CMake 3.31.6; executable reports ABACUS v3.11.0-beta9.
• Input: tests/08_EXX/08_KP_HSE_symm, preserving its SCF/EXX thresholds and three EXX outer updates.
• SHA256 verification covered 2664 source/CMake files before building. Variant patches, source hashes, binary hashes, logs, and matrix records were retained.
• All six cases exited 0 and produced #SCF IS CONVERGED#, final energy, and normal termination records.
• The original source was subsequently restored, hash-verified, and rebuilt.

The retained focused regression coverage includes:

• Poisoned-lower-triangle tests against an independent LAPACK reference: previous native ELPA 3/3 failed, corrected implementation 3/3 passed; previous real/complex cuSolver 2/2 failed, corrected implementation 2/2 passed.
• Caller-owned native ELPA H/S buffers remain unchanged across two consecutive solves.
• Two complex ELPA upper-triangle/fallback tests, three native ELPA block-size tests, and three K-point solver tests passed in the earlier merged-source validation.
• Twelve EXX symmetry tests passed, including little-group/star-weight restoration, representative independence, separate spin channels, and spinor-density fixtures.

Checks not run, with reason

• GPU tests were not rerun in this CPU six-way experiment; the earlier cuSolver positive/negative-control results are retained.
• The complete solver CTest and four-rank complex ELPA runs previously encountered failures inside the installed two-stage ELPA library. They are not counted as passing verification.
• Full-repository testing was not run; verification targets the changed solver contract and EXX restoration behavior.

What's changed?

This PR fixes two related correctness issues: Hermitian matrix input handling in native ELPA/cuSolver, and missing unitary little-group averaging before EXX k-star restoration.

1. Respect the authoritative upper triangles of H and S

For the generalized Hermitian eigenproblem,

LCAO supplies authoritative upper triangles of H/S. A solver must not depend on stale or unspecified lower-triangle entries.

The changes:

• Reconstruct private Hermitian H/S copies from the upper triangles before complex native ELPA solves.
• Preserve caller-owned buffers and avoid reusing a decomposition of temporary overlap storage.
• Select the upper triangle for real and complex cuSolver.
• Retain upstream's genelpa Hermitian-transform and fallback corrections.
• Correct the LAPACK regression workspace allocation to match its advertised size.

2. Average the IBZ density over its little group before star expansion

A k-star stores one representative operation for each distinct k point. It does not contain every operation fixing the irreducible k point modulo a reciprocal lattice vector.

Define the unitary little group as

The density must first be projected onto the invariant subspace:

In ABACUS's transposed density storage convention,

where $M_{g}$ is the AO rotation matrix in the code's convention.

This group average preserves Hermiticity, is idempotent, and makes the density invariant under little-group operations. Operations reaching the same star member differ by a little-group operation, so averaging removes dependence on the selected representative.

The implementation collects the required unitary little-group operations and AO rotations, averages the IBZ density, and then performs the existing star expansion. The existing star-size weight is applied separately after averaging. Identity-only little groups bypass the additional averaging work.

A single valid rotation preserves Hermiticity. The missing average instead leaves the density inconsistent with the symmetry relations assumed by reduced EXX contractions, which can produce a non-Hermitian Hamiltonian.

The six-way experiment demonstrates the consequence directly: little-group averaging removes the solver-dependent energy split and restores the full Hamiltonian's Hermiticity and eigenproblem residuals to roundoff. Matching the historical reference energy alone would not establish correctness.

The HSE reference and explanatory README are updated using independently checked results. The original case inputs and tolerances are retained.

Governance Notes

• INPUT/docs changes: no new parameter names or defaults. The existing EXX symmetry description is synchronized between its C++ registration, docs/parameters.yaml, and docs/advanced/input_files/input-main.md. The HSE reference and README document the corrected behavior.
• Core module impact: native ELPA input preparation, cuSolver triangle selection, EXX AO-rotation preparation and density restoration, and their regression helpers. The average covers unitary little-group operations; existing antiunitary restoration handling is retained. No new full magnetic-group projection algorithm or gga_grad functionality is introduced.
• Attribution: TaoXia's upstream genelpa corrections retain their original attribution. The native ELPA/cuSolver corrections, regression-helper changes, and EXX little-group restoration fix are included in this PR.
• Exceptions requested: none. Local governance checks report no blockers. The additional PARAM reference in the EXX test fixture is balanced by a removed production reference, giving net zero global-reference growth.